Reinforcement Learning of Parameters for Humanoid Rhythmic Walking based on Visual Information

This paper presents a method for learning the parameters of rhythmic walking to generate a purposive motion. The controller consists of the two layers. Rhythmic walking is realized by the lower layer controller which adjusts the speed of the phase on the desired trajectory depending on the sensor information. The upper layer controller learns (1) the feasible parameter sets that enable a stable walking for a robot, (2) the causal relationship between the walking parameters to be given to the lower layer controller and the change of the sensor information, and (3) the feasible rhythmic walking parameters by reinforcement learning so that a robot can reach to the goal based on the visual information. The method was examined in the real robot, and it learns to reach the ball and to shoot it into the goal in the context of RoboCupSoccer competition.